v0.4: historical trends — leaderboard + sparklines + headroom overflow projection#3
Merged
Merged
Conversation
…droom overflow projection v0.2 shipped per-build sizes shards; v0.3 added the Kconfig configurator; v0.4 finally reads across the retention window. The Drift tab catches "package X grew between two specific nightlies" — useful but you need to already suspect a regression to pick the right two builds. TrendsView watches the slope: if a package has been growing 4 KB/week for the last 14 days, it surfaces at the top of a leaderboard before the rootfs cap overflows. This is the direct answer to PR OpenIPC/firmware#2163's pain point — "~12 KB of accumulated drift since 2026-05-17 has now tipped it over." With historical trends, the bisect step that PR had to do becomes visible from the dashboard. Data shape scripts/prebuild.mts now walks every (build × platform) sizes shard once per source after the per-tag download loop, accumulates per-platform series into one PlatformAccumulator, and writes a per-platform aggregated file at: public/data/<source>/trends/trends.<platform>.json Schema (matches src/lib/timeseries.ts TrendsFile verbatim): { schema: 1, source, platform, generated_at, packages: { <name>: [{build_id, built_at, bytes}, ...] }, modules: { <name>: [{build_id, built_at, bytes}, ...] }, headroom_rootfs: [{build_id, built_at, used_kb, cap_kb, headroom_kb}, ...], headroom_kernel: [{...}, ...] } Every series ships sorted by built_at ascending. Defensive against partial shards: missing `headroom` blocks are skipped per build, not fatal. New FsHooks.read so the test memFs can serve readback inside the same in-memory state the gh-download mock writes into. Per-platform file: ~16 KB raw / ~1.5 KB gzipped on a board with 31 packages × 2 builds of data. Storage scales linearly with retention; ~80 KB raw at full 90-build retention. Total dist/data/firmware/trends/ across all 96 platforms today: 1.8 MB raw. Well under the plan's ~4 MB-gzipped envelope. UI TrendsView (new "Trends" tab between "Drift vs another build" and "Configure (what-if)"): * Header summary: builds count + date range covered * Two HeadroomChart sections — rootfs and kernel — that draw the used-bytes curve against the cap line. When projectOverflow(...) returns a non-null projection within 60 days, an amber badge surfaces "projected overflow in 7d (2026-06-12)". Linear regression: minimise (kb - (slope*day + intercept))^2, solve for kb=0. Sub-zero slope only — flat/growing returns null (no overflow projected). * Controls strip: window selector (7/14/30/90d), packages|modules toggle, "min weekly delta KB" filter to surface only the noteworthy growers. * Growers leaderboard sorted by absolute byte delta in window, newest 20. Each row gets an inline Sparkline (180×28 SVG polyline, colour by direction: amber=grow, green=shrink), plus a click-to-expand large sparkline (760×140) with axis labels. Sparklines are hand-rolled SVG — no chart lib. d3-hierarchy is already in the bundle (treemap) but doesn't transitively ship d3-scale/d3-shape; rather than add ~20 KB of charting deps for polylines and axes, the component pads + scales manually. Drop-in swap to a charting lib is easy if interactive zoom becomes a thing later. Tests tests/trends.test.ts (12 cases): * growthInWindow: window filtering, sort-on-input-order tolerance, first/last point semantics, < 2 points → null * bytesPerDay: window-endpoint slope, insufficient data → null * topGrowers: absolute-delta ordering, limit, skip insufficient, per-day rate * projectOverflow: flat/growing returns null, shrinking-at-known rate gives the expected days-to-zero tests/bundle.test.ts gains the same-origin invariant for trends/...json URLs — the regression guard for the original v0.1 CORS bug class now covers trends URLs too. tests/prebuild.test.ts memFs updated with `read` so the runPrebuild cases still pass; defensive `headroom?.rootfs` guards in emitTrends let the minimal-shard fixtures run through trends emission without exploding. Numbers - 63 active tests (was 46 in v0.3, +12 trends + adjustments) - JS bundle: 181 KB raw / 58 KB gzipped (+9 KB vs v0.3; under 250 KB perf budget) - Initial launch byte budget unchanged: ~60 KB gzipped (trends are lazy-loaded on Trends-tab click, ~1.5 KB gzipped per platform) - 96 trends files emitted today; 1.8 MB raw total Roadmap from README v0.4 (this) shipped → v0.5 (build-request flow, optional, depends on v0.3 fragment) remains. Maintenance backlog (cache eviction, 404 UX, drift URL state, picker search, data-window header indicator) unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v0.2 shipped per-build sizes shards; v0.3 added the Kconfig configurator;
v0.4 finally reads across the retention window. Drift catches "package X
grew between two specific nightlies" — useful but you need to already
suspect a regression to pick the right two builds. TrendsView watches
the slope — if a package has been growing 4 KB/week for 14 days, it
surfaces at the top of a leaderboard before the rootfs cap overflows.
Direct answer to PR OpenIPC/firmware#2163's
pain point — "~12 KB of accumulated drift since 2026-05-17 has now tipped
it over." With historical trends, the bisect step that PR had to do becomes
visible from the dashboard.
Data shape
`scripts/prebuild.mts` walks every (build × platform) sizes shard once
per source after the per-tag download loop, accumulates per-platform
series, writes:
```/trends/trends..json
public/data/
```
Schema (matches `TrendsFile` in `src/lib/timeseries.ts` verbatim):
```ts
{
schema: 1,
source, platform, generated_at,
packages: { : [{build_id, built_at, bytes}, ...] },
modules: { : [{build_id, built_at, bytes}, ...] },
headroom_rootfs: [{build_id, built_at, used_kb, cap_kb, headroom_kb}, ...],
headroom_kernel: [{...}, ...]
}
```
Series sorted ascending by `built_at`. Defensive against partial shards
(missing `headroom` is skipped per-build, not fatal). `FsHooks` gains
`read` so the test `memFs` can serve readback inside the same in-memory
state the gh-download mock writes into.
Per-platform file at full retention: ~80 KB raw / ~10 KB gzipped (plan
estimate). Today's ~2-nightly state: ~16 KB raw / ~1.5 KB gzipped per
platform.
UI
A new Trends tab between Drift and Configure:
against the cap line. When `projectOverflow(...)` returns a non-null
projection within 60 days, an amber badge surfaces
"projected overflow in 7d (2026-06-12)". Linear regression: minimise
`(kb − (slope·day + intercept))²`, solve for `kb=0`. Sub-zero slope
only — flat/growing returns null.
toggle, "min weekly Δ KB" filter.
selected window. Inline 180×28 SVG sparkline per row (amber=grow,
green=shrink), click row → expand 760×140 sparkline with axis labels.
Sparklines are hand-rolled SVG — no chart lib. `d3-hierarchy` is
already in the bundle (treemap) but doesn't transitively ship
`d3-scale` / `d3-shape`; rather than add ~20 KB of charting deps for
polylines, components pad + scale manually. Drop-in swap to a charting
lib stays easy if interactive zoom becomes a thing.
Tests
`tests/trends.test.ts` (12 cases):
semantics, < 2 points → null
per-day rate
rate gives expected days-to-zero
`tests/bundle.test.ts` extension: no JS chunk embeds
`releases/download/...trends...` URLs (same regression guard as v0.3's
kconfig assertion).
`tests/prebuild.test.ts` memFs updated with `read`; defensive guards
in `emitTrends` let minimal-shard fixtures pass.
Numbers
Under the 250 KB perf budget.
Test plan
emitted
× 2 builds of data, headroom_rootfs has 2 entries
correctly rejected by main-only branch policy) before merge
production state
richer as retention fills toward the ~2026-09-01 90-day mark
🤖 Generated with Claude Code